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Executive Summary 


This report analyses the general trends in the data across several studies investigating the impact 
of Covid-19 on educational attainment in England, from March 2020 onwards, and identifies where 
particular attention is needed. It is broken down into four research questions which consider the 
impact of Covid-19 by: 


) pupil age 

) disadvantaged and non-disadvantaged pupil status 
3) prior pupil attainment levels 

) school subject (reading and mathematics). 


Amidst concern about the pandemic’s possible impact on the development of children and young 
people’s social and emotional skills and their wellbeing, there has been some research in this 
important area. This report, however, focuses on pupils’ attainment. 


Despite the reduction in the progress of pupil learning, which the media have reported as the 
Covid-19 “learning loss” (referred to here as the “Covid gap”), there have been some signs of 
recovery in the latter part of the 2020-2021 school year. 


Looking initially at school phase, there is a lack of evidence concerning the impact of the pandemic 
on secondary-aged pupils, in particular Key Stage 3. The National Reference Test provides some 
information about Year 11 attainment in English and mathematics, but there is very little beyond 
that year group or those subjects. However, for primary-aged pupils it is clear that all year groups 
performed at a lower level than expected in Autumn 2020, in both reading and mathematics. By 
Spring 2021, there was a further drop in attainment, especially in the younger year groups, 
following the partial school closures in January-March 2021. By the Summer of 2021, however, all 
years saw the Covid gap become narrower. The negative impacts of the pandemic on reading 
were greatest in Key Stage 1 (and in Year 1 in particular), whilst in Key Stage 2 it was mathematics 
attainment that was impacted the most and which also showed slower recovery than reading. 


When looking at the data relating to disadvantaged pupils, it is important to remember that the 
disadvantage gap prior to Covid-19 was at least twice as large as any impact of Covid-19 on pupil 
attainment. In Autumn 2020, the disadvantage gap was wider than it was pre-pandemic, indicating 
that the initial lockdown had a greater impact on disadvantaged pupils than non-disadvantaged 
pupils. However, since the initial period of partial school closures there is no consistent evidence of 
a further widening of this gap, suggesting that disadvantaged pupils are recovering at around the 
same rate as non-disadvantaged pupils. This is despite the targeting of recovery funds and 
initiatives on disadvantaged pupils which might have been expected to enable them to recover at a 
faster rate. 


The distribution of attainment has shifted as a result of the disruption caused by Covid-19. By 
Summer 2021, in reading, more pupils and particularly those in the younger age groups, are 
scoring low marks on the assessments, whilst in mathematics fewer pupils are getting high scores 
in Key Stage 2. 


Studies find the impact on achievement is, on average, similar in both reading and mathematics, 
with the two periods of partial school closures (March-June 2020 and January-March 2021) 


The Impact of Covid-19 on pupil attainment 


:*NFER 


National Foundation for 
Educational Research 


associated with less progress in both subjects and with evidence that each is followed by a period 
of some recovery. However, as noted above, there are different patterns by age. The impact on 
progress in reading has been greatest in Years 1 and 2 while the impact of the pandemic on 
mathematics progress is greater across Key Stage 2. 


The evidence suggests that interventions and resources would be best targeted at the 
development of reading in Key Stage 1 and identifying areas of the Key Stage 2 mathematics 
curriculum with which pupils are struggling. Additionally there should be renewed focus on 
reducing the disadvantage gap. 


There is a need for careful monitoring of and research into the longer term effects of the disruption 
to learning of the pandemic in secondary as well as primary schools, to determine the extent to 
which targeted measures are effectively supporting the schools and pupils most affected. With our 
focus on attainment, in this report we have not looked at evidence from a number of studies which 
are exploring the wider impact of the pandemic on pupils’ wellbeing and mental health. 
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1 Introduction 


The pandemic has led to considerable disruption to the learning of all pupils in England. There 
have been two periods in which schools have been closed to a majority of pupils — the first period 
of partial school closures of March—June 2020, and the second of January—March 2021. Individual 
schools have also had to manage a variety of different levels of disruption in the intervening 
months and many secondary-aged students did not return to school after June 2020 until 
September. 


In order to inform policy- and decision-making, a range of studies have used standardised test data 
to estimate the impact of this disruption on pupils’ learning. This primarily involves comparing 
attainment during the pandemic with attainment prior to the pandemic. We have drawn evidence 
from across these studies and further detail is contained in the appendix. 


There is strong evidence that the disruption to schooling in England between March and 
September 2020, including the first period of partial school closures (March—June 2020), led to a 
slowing of progress for primary-aged pupils: the creation of what is termed the ‘Covid gap’. 


Standardised test data from the end of the Autumn term 2020 indicates a narrowing of the Covid 
gap (DfE, 2021; Blainey et a/., 2021a). A second time of partial school closures from January to 
March 2021, with schools providing different degrees of online learning, then resulted in a widening 
of the Covid gap, with pupils making less progress during this period than is typically seen. On 
return to school and until the end of the 2020-21 academic year, there is evidence of some 
recovery: a narrowing of the Covid gap (Blainey ef a/., 2021b; DfE, 2021; Rose et a/., 2021). 


For most year groups, gaps persisted to some extent through into Summer 2021. There is some 
variability according to pupil age and also by subject (reading or mathematics). This is shown in 
Figures 1 and 2 below which draw together the evidence from Rose et al. (2021) and Blainey et ai. 
(2021b). Other research, including DfE (2021), uses different methods to calculate the scale of the 
Covid gap. 


This report looks at the available evidence about the scale of the Covid gap and whether specific 
groups of pupils and curriculum areas have been impacted to a different extent by the pandemic. 
Our main focus is on the impact in primary schools, as most of the evidence is drawn from this age 
group’s attainment. 
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Figure 1: Covid gap by primary school year for reading 
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Figure 2: Covid gap by primary school year for mathematics 
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2 How has Covid-19 impacted the attainment of pupils of 
different ages? 


IMPACT 


By late Autumn 2020, all primary year groups were showing less progress than in previous years in 
both reading and mathematics. 


There was a further impact in both Key Stages 1 and 2 following the Spring 2021 partial school 
closures with some evidence that the reading attainment of the youngest children was particularly 
negatively affected. 


RECOVERY 


Whilst there were signs of recovery across all year groups by Summer 2021, there was some 
unevenness by age. 


In Key Stage 1, there were limited signs of recovery with different studies noting greater recovery 
in different subjects. There was consistently a greater negative impact of the pandemic on Year 1 
pupils. This emphasises the need for time and resources to be focused on the early literacy and 
numeracy skills of the youngest children at this critical stage in their development. 


For Key Stage 2, recovery by Summer 2021 was greater for reading than mathematics. 


2.1 Key Stage 1 summary findings 


Data on Key Stage 1 attainment is drawn from four sources: Blainey et a/. (2021a and 2021b), DfE 
(2021) and Rose et al. (2021). When performance in Autumn 2020 is compared to pre-pandemic 
performance, there is clear evidence of the impact of the disruption to schooling caused by Covid- 
19 on the youngest learners, resulting in a slowing of progress. 


In a number of studies, there is an interim measure of attainment in Spring 2021, after the second 
national lockdown, school closures and online teaching (January—March 2021). Where there is 
separate evidence for Key Stage 1, this consistently shows a widening of the Covid gap by Spring 
2021 (Rose et al., 2021; Blainey et al., 2021b). 


The final data comes from assessments undertaken in Summer 2021 and follows several months 
of in-person teaching for the vast majority of pupils. At this point, there is evidence of progress in 
both year groups and subjects, but a Covid gap remained. Blainey et a/. (2021b) recorded that for 
reading, across all the primary years, the gap was widest for Year 1 pupils. Rose ef al. (2021) also 
noted the extent to which Year 1 pupils were performing below pre-pandemic levels. Evidence for 
recovery in mathematics was mixed with Blainey et a/. (2021b) recording the widest Covid gaps in 
data from reception, Year 2 and Year 6. while Rose et al. (2021) found Year 2 mathematics 
achievement to be similar to pre-pandemic levels. 


Another source of evidence is provided by Nash et al. (2022) who investigated the impact of the 
pandemic on even younger children’s development by looking at progress against the Early 
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Learning Goals. This small study (n = 10 schools and 450 children in Leeds) analysed assessment 
data from teachers of children who were in reception at the point of the first lockdown in March 
2020. This sample of pupils was then tracked throughout Year 1 (2020/21). At the end of Year 1,a 
year later than the usual assessment, 19 per cent of the sample had still not yet reached the 
standard expected for the end of the reception year. 


Finally for this age group, there is some limited data about their writing attainment. Using the 
comparative judgment methodology, Christodoulou (2022) reported evidence of a Covid gap in the 
achievement of Year 1 pupils in January 2022, comparing it to January 2020. This was largely due 
to a higher proportion of pupils achieving low scores. 


2.2 Key Stage 2 summary findings 


Data on the impact of the pandemic on Key Stage 2 pupils’ reading and mathematics attainment 
comes from two suites of standardised tests: Rising Stars (Blainey et a/., 2021b) and Renaissance 
Learning (DfE, 2021). 


Blainey et al. (2021b) reported evidence of the widest Covid gap in reading being seen in the 
Spring term assessments in 2021 i.e. after the second phase of partial school closure, for each 
year group. There is then evidence of recovery by summer 2021, with the greatest recovery in 
Year 6, where attainment returned to close to pre-pandemic levels. In all year groups, in 
mathematics, the Covid gap is widest in the Spring 2021 assessments but, compared to reading, 
there is less recovery evident by the time of the Summer term assessments. Blainey et al. found 
least change in the Covid gap between the Spring and Summer terms in Year 6. 


The DfE data set is not represented in Figures 1 and 2 above as the unit of measure is different to 
that used by Rose et al. (2021) and Blainey et al. (2021b). However, the patterns in the data are 
broadly similar with the widest Covid gaps after periods of partial school closure followed by some 
recovery. By Summer 2021, the Covid gap remained wide for mathematics but there were signs of 
recovery. In reading, recovery during the Summer term appeared to be faster than for 
mathematics. 


Christodoulou (2021b) reported evidence of Year 5 writing attainment, having dipped in November 
2020, being back to 2019 levels by November 2021. Similarly, by October 2021, on average, Year 
3 writing was at slightly above October 2019 and 2020 levels, although the distribution showed a 
wider spread of achievement with greater proportions of pupils achieving high and low scores than 
previously (Christodoulou, 2021a). 


2.3. Key stages 3 and 4 summary findings 


Evidence for the impact of Covid-19 on secondary-aged pupils is from two sources: DfE (2021) and 
Burge et al. (2021). 


DfE, using Renaissance Learning data for reading only, reported a Covid gap for Key Stage 3 
pupils in the first half of the Autumn term 2020 that was smaller than that seen for primary school 
pupils at this time. However, by the end of the 2020/21 school year there was less evidence of 
recovery for Key Stage 3 and at this point the Covid gap was greater than for primary-aged pupils. 


The National Reference Test (Burge et a/., 2021) provided evidence of attainment in English and 
mathematics in Year 11 from just prior to the first lockdown (pre-pandemic) and compared this with 
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achievement in Summer 2021 (mid-pandemic). English results in 2020 and 2021 were broadly 
similar. However, mathematics performance in Summer 2021 on this monitoring assessment 
showed a statistically significant drop from the previous year at the three grade boundaries (4, 5 
and 7) that are monitored. 
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3 Has the pandemic had a disproportionate impact on 
disadvantaged pupils? 


IMPACT 


Before the pandemic, there was a large gap in the proportions reaching the expected achievement 
standards in English primary schools between disadvantaged and non-disadvantaged pupils. This 
‘disadvantage gap’ is wider than any gap resulting from the impact of the pandemic (the ‘Covid 
gap’). 

There is strong evidence that this disadvantage gap has widened during the pandemic i.e. the 
impact of the pandemic on the progress of disadvantaged pupils has been greater than on the 
progress of non-disadvantaged pupils. This widening appears to have been greater in mathematics 
than in reading in most primary year groups. 


RECOVERY 


By Summer 2021, there is some evidence that disadvantaged pupils are recovering at broadly the 
same pace as non-disadvantaged pupils. 


The scale of the disadvantage gap reminds us that some pupils were more vulnerable to the 
impacts of the pandemic. This reinforces the need for policy-makers to renew their efforts to 
address this long-standing feature of the education system. 


3.1 Pre-pandemic 


There is established evidence of an attainment gap between disadvantaged and non- 
disadvantaged pupils. For example, Hutchinson et a/. (2020) record that disadvantaged pupils 
were, on average, more than nine months behind their peers at the end of their primary schooling 
in 2019. This is larger than any deficit reported as a consequence of the pandemic. Weidmann et 
al. (2022) report slightly narrower but nonetheless substantial gaps for reading and mathematics 
based on data from Year 2 to Year 6 in Autumn 2019. 


In the last year for which national curriculum statutory assessment data is available (DfE, 2019b), 
there was a 17 percentage point gap in reading at Key Stage 1 between the proportion of 
disadvantaged pupils who reached or exceeded the expected standard (teacher assessment) and 
all other pupils. The equivalent figure for mathematics was also 17 percentage points. 


In Key Stage 2 for 2019 (DfE, 2019a), results of national curriculum statutory assessments showed 
a 16 percentage point gap in reading between the proportion of disadvantaged pupils who reached 
or exceeded the expected standard and all other pupils. At this Key Stage, the equivalent figure for 
mathematics was 17 percentage points. 


In primary school, looking back over previous years, there is some evidence that the disadvantage 
gap, defined as the proportion of pupils achieving the expected standard in reading, writing and 
mathematics, was narrowing, albeit very slowly. However, progress appeared to have stalled or 
even reversed in 2019 (Hutchinson et a/., 2020). 
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3.2 Impact of the pandemic 


All sources of evidence included in this paper find large gaps between the achievement of 
disadvantaged and non-disadvantaged pupils throughout the pandemic and some evidence that 
this has increased as a consequence of the pandemic. 


Research with pupils in Key Stage 1 (Rose et a/., 2021) noted that the disadvantage gap peaked in 
the Spring 2021 assessments, while Christodoulou (2022) found a greater fall in the writing 
performance of disadvantaged pupils in Year 1 in January 2022 compared to non-disadvantaged 
pupils, indicating a further widening of the gap. 


Weidmann et al. (2022) analysed Rising Stars reading and mathematics data with the same 
sample of pupils (in Years 2 to 6) and across five time points. The authors conclude that between 
the start of the pandemic and Summer 2021 there was an increase in the disadvantage gap for 
mathematics (of about 11 per cent) but not for reading. This gap appeared to have widened most 
in the early months of the pandemic up to September 2020. With regard to a possible relationship 
with age, they did not find clear differences between the year groups they studied, although the 
strongest evidence for widening disadvantage gaps in mathematics was in Years 2 and 3. 


Blainey et al. (2021b), also with Rising Stars data although a different data set, reported that in 
Summer 2021 the widest disadvantage gap by year group was for Year 1 in reading and for Year 6 
in mathematics. When the gaps are compared with what was seen in Summer 2019, the largest 
increases in the disadvantage gaps were seen in Year 1 (reading) and Year 4 (mathematics). 


DfE (2021), analysing Renaissance Learning data, reported that the disadvantage gap increased 
during the pandemic in reading in primary and secondary schools. It also increased in mathematics 
in primary schools (there is no mathematics data available for secondary schools) and the increase 
in mathematics was greater than that seen in reading. 


Christodoulou (2021a) found in Autumn 2020 and 2021 that the writing performance of non- 
disadvantaged pupils in Year 3 had increased year on year when compared to pre-pandemic 
achievement (Autumn 2019), whereas that of disadvantaged pupils had fallen in 2020 and then 
recovered to pre-pandemic levels by 2021. Due to the improvement in the performance of non- 
disadvantaged pupils, the gap between the groups has widened during the pandemic. 


3.3. Recovery 


There is some evidence that disadvantaged pupils are recovering at broadly the same rate as non- 
disadvantaged pupils but more evidence is needed to draw clear conclusions about the patterns. 


Rose et al. (2021) found that the gap between disadvantaged and non-disadvantaged pupils in 
Year 1 had reduced between Spring and Summer 2021 in both reading and mathematics, although 
it remained substantial. 


In the DfE (2021) research, the gap, which had increased earlier in the pandemic between the 
achievement of primary-aged disadvantaged and non-disadvantaged pupils, showed some 
narrowing in Summer 2021, and both groups showed some signs of recovery. By the end of the 
school year, however, the disadvantage gap remained wider than it had been at the start of the 
pandemic. 
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3.4 Area-level deprivation 


Weidmann et al. (2022) noted that disadvantage has an impact not only at pupil-level but also at 
area-level: non-disadvantaged pupils in schools in areas of medium and high levels of deprivation 
experienced a similar negative impact of the pandemic as disadvantaged pupils in areas of low 
deprivation. 


Lucas et al. (2020) used a teacher survey in Summer 2020 in primary and secondary schools to 
explore the association between the level of deprivation of the school and pupil engagement. They 
found that pupil engagement, including disadvantaged pupil engagement, was likely to be lower in 
schools with the highest levels of deprivation. They also found that disadvantaged pupils in schools 
in the most deprived areas were less likely to be highly engaged than their peers, compared with 
disadvantaged pupils in schools in the most affluent areas. Finally, low pre-pandemic attainment 
was associated with lower levels of engagement. 


3.5 Regional impact 


There is some evidence of regional disparities in the impact of Covid-19 across the country. The 
English regions have not been consistently defined in different studies and as a consequence the 
evidence is limited at this time. In addition, not all differences between regions are significant due 
to small sample sizes. The data available (DfE, 2021) indicates a broadly similar pattern across 
regions with a gap that narrowed in the Summer term of 2021 compared to the Spring term for 
reading and mathematics. By the end of the 2020/21 school year, in all regions except the East 
and West Midlands (reading) and the East Midlands (mathematics), the Covid gap was narrower 
than it had been in the first half of the Autumn term 2020. 
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4 Is the impact of Covid-19 on attainment evident across all 
ability levels? 


IMPACT 


Over the 2020/21 school year, the data shows that there was a tendency for more pupils to 
achieve scores at the lower levels of the distribution; this was especially evident in the data from 
younger pupils in reading and writing. 


RECOVERY 


Despite some overall signs of recovery, by Summer 2021 there remained a higher than expected 
proportion of lower scoring pupils in Year 1 reading and mathematics. This was also the case for 
Year 2 reading. 


These are the children who are at risk of future educational underachievement. This evidence 
emphasises the urgency of addressing this issue through focused input and adequate resourcing 
before these children become struggling or reluctant readers. 


Rose et al. (2021) reported much higher proportions of pupils in Years 1 and 2 scoring at the lower 
end of the score range than seen before the pandemic. This was particularly evident in reading and 
has significant educational implications which are discussed in our conclusion. Whilst some 
recovery was evident in Summer 2021 particularly in Year 2, there remained a higher proportion of 
lower attainers than would be expected, based on the pre-pandemic distribution for the 
standardisation. 


A similar picture was seen in Blainey et a/. (2021b) in reading data in the Summer term of 2021 for 
the youngest pupils. At this time, however, Year 6 pupils were showing the range of reading 
achievement typical for pre-pandemic times. 


Repeating the pattern seen for reading, although slightly less marked, in both Spring and Summer 
2021, Rose et al. (2021) found a higher proportion of children scoring at the lower end of the score 
range for mathematics in Year 1 than would be expected based on the standardisation of these 
assessments. This mirrored the picture for Year 2 mathematics in Autumn 2020 and Spring 2021. 


Blainey et al. (2021b) reported the mathematics scores of Year 6 pupils in the middle of the 
distribution showing the greatest dip in Summer 2021. There was no evidence provided for other 
year groups. 


It is worth noting that for all of these data points, proportions scoring at the upper end remained 
largely unchanged. There were still pupils scoring around maximum marks. 


The limited evidence available for writing (Christodoulou, 2022) also showed that in Year 1 in 
January 2022 there was an increase in the proportion of lower scoring pupils with no change in the 
proportion with high scores. In Year 3 writing in Autumn 2021, the spread of scores had increased 
with more pupils scoring at each end of the distribution and fewer in the middle (Christodoulou, 
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2021a). Year 5 writing in Autumn 2021 showed no change in the pattern of mark distribution seen 
in previous years (Christodoulou, 2021b). 
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5 Has pupils’ progress in reading and mathematics been 
equally affected by Covid-19? 


IMPACT 


The two periods of partial school closures were associated with less progress in both reading and 
mathematics and there is some evidence that each was followed by a period of recovery. 


By Summer 2021, the evidence suggests that in Years 1 and 2, the greater impact was on reading 
progress, and, across Key Stage 2, on mathematics progress. 


These patterns give some indication as to where schools may need to focus additional time and 
resource in order to address particular gaps. 


5.1 Impact of the pandemic 


The repeated measures analysis by Rose et al. (2021) showed that, between late Autumn 2020 
and late Spring 2021, progress of Year 2 pupils in both reading and mathematics slowed i.e. the 
Covid gap widened. 


Blainey et al. (2021b) reported the Covid gap at its widest in the Spring 2021 assessment window 
in both subjects and consistently wider for mathematics than reading in Years 3 to 6. 


For Year 3 to Year 6, DfE (2021) found a wider Covid gap in mathematics than in reading at each 
assessment point during the school year 2020/21, with the gap between the two subjects widest in 
the first half of the Autumn term. 


Data from GL Assessment (Brzyska et al., 2021) also indicated that the Covid gap was wider in 
mathematics assessment outcomes than in reading by the end of 2020, with data from primary- 
and secondary-aged pupils. 


There is limited data available looking at the attainment of secondary-aged pupils, but the data 
from the National Reference Test involving a sample of Year 11 pupils (Burge ef al., 2021) 
indicates that while there was no change in the performance in English, there was a significantly 
lower performance in mathematics in 2021 compared to 2020 (pre-pandemic). 


5.2 Recovery 


In reading in Year 1, Rose et als (2021) repeated measures analysis showed that on average, 
between Spring 2021 and Summer 2021, pupils made expected progress i.e. the Covid gap did not 
change. In contrast, between Spring and Summer 2021, there was evidence of faster progress 
than expected in Year 1 mathematics leading to a narrowing in the Covid mathematics attainment 
gap, providing evidence of recovery. 


Blainey et al. (2021b) report that by the Summer of 2021, the reading of Year 1 pupils was showing 
a greater impact from the disruption than their mathematics while for Years 2 to 6, the impact was 
greater in mathematics. DfE (2021) data also shows that mathematics attainment in Years 3 to 6 
was impacted more than that of reading although there were clear signs of recovery in both 
subjects in the Summer of 2021. 
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The structure of the mathematics curriculum is quite different to that of reading which revisits and 
develops broadly the same skills in each school year. This does of course happen, to some extent, 
in mathematics but there is a greater requirement for new content to be introduced through each 
year. The teaching time lost due to school partial closures will clearly have had a large impact on 
the amount of time teachers have had available to deliver this new content in mathematics and it is 
possible that many teachers focused on consolidating learning in curriculum areas already taught 
rather than tackling new topics. Eivers et a/. (2020) reported that most lessons were offline in the 
first period of partial school closures. Additionally, it can be more difficult, when teaching remotely, 
to identify pupils’ misconceptions and barriers to understanding in a timely manner. This makes it 
harder to provide the support that pupils need to progress and allows the misconceptions to 
embed. 
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6 Conclusion 


This report summarises the impact on pupils’ achievement of an exceptional period in our lifetimes 
and points to some areas which need particular attention. From this overview, several recurring 
themes emerge: 


e The progress of the youngest children has been particularly affected, especially in relation to 
their reading development. 

e For slightly older pupils, in Key Stage 2, by the Summer of 2021 it was clear that mathematics 
progress continued to be affected. 

e There are clear indications of recovery across both key stages and subjects but the extent 
varies. 

e The disadvantage gap has widened during the pandemic and is wider than any gap created by 
Covid-19. 


It is important to acknowledge that during this unprecedented period of disruption in their school 
lives, children were learning. Assessment data tells us that progress was often less than that seen 
pre-pandemic, but children were still moving on with their learning and in some cases we are 
already seeing achievement return to close to where we would expect it to be. But not in all cases. 


Of particular concern is the impact on the development of early literacy skills by the youngest 
pupils in primary school. The proportion of children who struggled to engage with the reading 
assessment in Year 1 more than doubled in Summer 2021 compared to the pre-pandemic sample 
(Rose et a/., 2021). We would have expected to see around 16 per cent of pupils achieving a 
standardised score of 85 or less. In Summer 2021, 27 per cent of Year 1 children in the nationally 
representative sample scored in this low band. 


Early reading plays a key part in children’s later achievement. This is not only the skill of reading 
but also an engagement with literacy-related activities. Teachers endeavour to develop in their 
pupils the knowledge that we turn to reading and to books for both pleasure and information. They 
don’t want to see a child at the age of 5 or 6, believing that reading isn’t for them or that it’s 
something to be avoided: reading engagement and reading performance are mutually reinforcing. 


We know that some children are more at risk of lower achievement as a result of poverty. But we 
have evidence from PISA 2000 of how powerful high levels of engagement in reading activities can 
be: pupils whose parents had the lowest occupational status but who were highly engaged readers 
had better reading scores than pupils whose parents had high or medium occupational status but 
who had low levels of reading engagement (OECD, 2002). Similarly PISA 2018 found that 
academically resilient pupils (disadvantaged pupils performing in the top quarter of reading 
proficiency) tended to enjoy reading more (OECD, 2019). 


Teachers not only succeed in teaching most children to read, they have at their fingertips a 
multitude of imaginative ways to engage children in the world of books. In the current context, it is 
going to be even more important to ensure schools are adequately resourced for this, in order to 
reduce the risk that one consequence of the pandemic is an increase in the numbers of reluctant 
readers, along with the associated negative impact on their self-esteem and potentially behaviour. 


Whilst progress in mathematics learning in the primary phase slowed during the pandemic, it is 
reassuring to note some evidence of recovery. Online teaching required teachers to adopt new 
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pedagogies and we know that school leaders have had to explore new approaches to curriculum 
delivery (Nelson et a/., 2021). Some schools adopted a narrower curriculum during the pandemic, 
with a focus on key skills. Others focused on key components within a subject. One potential risk of 
these strategies is a less coherent curriculum. Mathematics learning is hierarchical and so gaps in 
learning may only emerge later. Adopting a diagnostic approach to mathematics assessment, 
involving close analysis of the errors pupils make with a view to identifying and then remediating 
gaps and misconceptions, has the potential to accelerate recovery. Resources from the National 
Centre for Excellence in the Teaching of Mathematics (NCETM) and the DfE’s Ready to Progress 
materials are valuable. 


Finally, whilst the focus of this report is on the evidence generated over the past two years about 
the extent of the Covid gap — the impact of the pandemic on primary-aged pupils’ attainment — it 
cannot be forgotten that another, larger, gap existed before the pandemic: the disadvantage gap. 
Pre-Covid, there were suggestions that attempts to narrow this gap and reduce inequalities had 
stalled (Hutchinson et a/., 2020) and in this report we conclude that in fact, Covid is likely to have 
contributed to its widening. In a post-Covid world, hopefully with greater knowledge of effective 
‘catch up’ strategies, urgent attention should return to reducing the inequalities in the education 
system. 
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Appendix 


Methodology 


The evidence sources for this report are provided by NFER’s Knowledge Management Team who 
have maintained a Covid-19 research and policy database throughout the pandemic. 


We focused on data from England only. This was for two main reasons: 


e education policy is devolved to the administrations of Northern Ireland, Scotland and Wales; 
e much of the evidence is drawn from standardised tests that are designed to cover the national 
curriculum used in England. 


Whilst we have not explicitly excluded evidence concerning secondary-aged pupils, it is clear that 
the majority of studies include either only the primary years, or include Key Stages 1 and 2 in much 
more detail than Key Stages 3 or 4. 


We have focused on reading and mathematics. These are two core subjects in the English national 
curriculum and the accountability system, and they form the basis of test publishers’ suites of 
primary assessments. In addition, there is some limited evidence emerging about the impact of the 
pandemic on pupils’ writing attainment. This has been included in the appropriate sections where 
available. It is important, though, to recognise that English and mathematics may have received 
particular attention during the pandemic (see for example, Nelson et a/, 2021); as a result, we 
caution against generalising from these findings to other subjects. 


The cut off for inclusion is a publication date before 16 February 2022. As a result of the desire to 
produce useful data for schools to inform planning and resource allocation, many reports have 
been produced with a degree of urgency. Most have not been subject to full peer review, and have 
possibly not been through the same academic rigour as papers produced under less time 
pressure. 


A summary of the data sources is shown in Table 1. 
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Table 1 Summary of data sources used in this report 


Source By age Disadvantage | By ability By subject 


Early learning goals 
Nash et al. (2022) 


v 


GL Assessment (reading & 
mathematics) v 
Brzyska et al. (2021) 


National Reference Test 
(English & mathematics) v v 
Burge et al. (2021) 


NFER (reading & 
mathematics) v v v v 
Rose et al. (2021) 


Renaissance Learning 


(reading & mathematics) v v v 
DfE (2021) 

Rising Stars (reading & 

mathematics) J Y y 2 


Blainey et al. (2021a; 2021b) 
Weidmann et al. (2022) 


Writing (comparative 
judgement) 
Christodoulou (2021a; 
2021b; 2022) 


Disadvantage 


There has been a particular interest in the impact of Covid-19 on the disadvantage gap. The 
studies included did not have a consistent approach to identifying disadvantage: some studies 
used pupil-level eligibility for free school meals (FSM) as a proxy for economic disadvantage; one 
used the area-level Income Deprivation Affecting Children Index (IDACI); another categorised 
schools according to the proportions of pupils eligible for FSM. Where it is available, we report data 
based on the first approach (eligibility for FSM). 


Role of teacher assessment data 


The main evidence in this report is a summary of findings from standardised tests administered at 
intervals throughout the pandemic. We have not included evidence of teachers’ estimates of the 
impact of Covid-19 on pupils’ attainment and have focused on norm- or criterion-referenced 
assessment outcomes. 


A report was published in February 2021 by Juniper Education with evidence derived from a large 
dataset. This data is teachers’ assessments of whether pupils have reached age-related 
expectations each term. As an agreed set of standards does not exist for termly age-related 
expectations, we excluded this evidence. The report may be useful for teachers as they plan for 
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their own classes but it did not meet our standards of norm- or criterion-referenced assessment 
data. 


In contrast, a small study by Leeds University looked at Early Years Foundation Stage data (Nash 
et al., 2022). Whilst this is also based on teacher assessment, these assessments are based ona 
common set of criteria known as the Early Learning Goals. 


Measurement 


Most of the sources of evidence that we have reviewed compare pupils’ progress during the 
pandemic with the rate of progress seen pre-pandemic. Sometimes the performance of individual 
pupils is tracked during the pandemic; in other datasets assumptions are made about the 
comparability of large and nationally representative samples comprised of different pupils. 


One unit of measurement commonly used to report the impact of the pandemic is months of 
progress (or lack of progress, often referred to as ‘learning loss’). This approach is also widely 
used in evaluations of the effectiveness of interventions and a methodology designed by the 
Education Endowment Foundation (EEF) includes a conversion table for effect size to months of 
progress. It is a powerful means of indicating the scale of the impact of an intervention and is more 
accessible than reporting effect sizes; however, we have elected not to use this approach for the 
following reasons: 


e The methodology adopted by EEF does not take into account the different rates of progress 
made by pupils of different ages, an important part of this report. 

e = It is arelatively blunt measure, with the EEF approach converting effect sizes (each of up to 2 
decimal places) within a range to complete months of progress. 

e One of our key sources of evidence (DfE, 2021) reported differences in progress in months but 
used a different methodology to arrive at these to that adopted by two other key sources: Rose 
et al., (2021) and Blainey et a/., (2021b). 

e Other sources (e.g. Brzyska et al., 2021; Nash et al., 2022) reported neither effect sizes nor 
months of progress. 


Our approach, therefore, has been to look for patterns within each study that contribute to an 
understanding of our research questions, and then to look for patterns between studies. 
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